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Abstract 

Given  a  number  of  patrollers  that  are  required  to  detect  an  intruder  in  a  channel,  the  channel 
patrol  problem  consists  of  determining  the  periodic  trajectories  that  the  patrollers  must  trace 
out  so  as  to  maximized  the  probability  of  detection  of  the  intruder.  We  formulate  this  problem 
as  an  optimal  control  problem.  We  assume  that  the  patrollers’  sensors  are  imperfect  and  that 
their  motions  are  subject  to  turn-rate  constraints,  and  that  the  intruder  travels  straight  down 
a  channel  with  constant  speed. 

Using  discretization  of  time  and  space,  we  approximate  the  optimal  control  problem  with 
a  large-scale  nonlinear  programming  problem  which  we  solve  to  obtain  an  approximately  sta¬ 
tionary  solution  and  a  corresponding  optimized  trajectory  for  each  patroller.  In  numerical  tests 
for  one,  two,  and  three  underwater  patrollers,  an  underwater  intruder,  different  trajectory  con¬ 
straints,  and  several  intruder  speeds,  we  obtain  new  insight  —  not  easily  obtained  using  simply 
geometric  calculations  —  into  efficient  patrol  trajectory  design  for  multiple  patrollers  in  a  nar¬ 
row  channel  where  interaction  between  the  patrollers  is  unavoidable  due  to  their  limited  turn 
rate. 
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1  Introduction 


This  paper  deals  with  the  optimal  detection  of  an  underwater  intruder  in  a  channel  using  one  or 
more  unmanned  underwater  vehicles  (UUVs).  In  particular,  it  establishes  optimal  periodic  patrol 
trajectories  for  the  UUVs,  which  we  refer  to  as  patrollers,  that  maximize  the  probability  of  detection 
of  an  underwater  intruder  traveling  straight  down  a  channel  at  constant  speed.  While  we  focus  on 
an  underwater  intruder  and  patrollers,  our  general  approach  may  also  be  applicable  in  the  case  of 
other  types  of  vehicles. 

This  problem  is  a  multi-patroller  extension  of  the  classical  “channel  patrol  problem”  (also 
called  the  barrier  patrol  problem);  see,  e.g.,  Section  1.3  of  [13]  and  Chapter  9  of  [12].  The  channel 
patrol  problem  for  a  single  patroller  was  formulated  by  Koopman  [7]  during  World  War  II  and 
arises  in  naval  operations  where  the  channel  may  represent  a  relatively  narrow  body  of  water  such 
as  a  strait  or  port  entrance  through  which  enemy  vessels  and  submarines  as  well  as  smugglers  and 
terrorists  may  attempt  to  pass.  The  need  to  consider  multiple  patrollers  is  apparent,  especially  in 
view  of  the  development  of  small  UUVs  that  may  be  used  to  guard  channels.  The  channel  patrol 
problem  may  also  arise  in  anti-submarine  warfare  in  an  operating  area  around  a  carrier  or  naval 
expeditionary  strike  group  [11]  and  then  typically  with  multiple  patrollers.  With  the  proliferation  of 
small  diesel  submarines  and  the  advent  of  UUVs  and  self-propelled  semi-submersibles  the  channel 
patrol  problem  has  acquired  new  importance,  since  these  vessels  are  difficult  to  detect.1 

The  early  studies  by  Koopman  [7]  as  well  as  by  Washburn  [14]  focus  on  the  determination 
of  the  probability  of  intruder  detection  for  a  single  patrol  trajectory  consisting  of  piecewise  linear 
segments;  see  also  Chapter  9  of  [12].  This  approach  results  in  simple  formulae  for  the  probability 
of  detection  and  provides  insight  into  the  effectiveness  of  “back-and-forth”  versus  “bow-tie”  tra¬ 
jectories  for  various  patroller  and  intruder  speeds.  In  reality,  a  vessel  cannot  carry  out  a  perfect 
back-and-forth  patrol  trajectory  as  it  is  unable  to  turn  around  instantaneously  at  the  end  of  each 
channel  crossing.  These  early  studies  ignore  the  limited  turn-radius  of  the  patroller  or  use  coarse 
approximations.  Moreover,  they  focus  on  a  single  patroller  with  the  assumption  that  the  case  of 
multiple  patrollers  can  be  solved  by  dividing  the  channel  into  subchannels,  with  one  patroller  as¬ 
signed  to  each  subchannel.  This  policy  may  become  problematic  when  there  are  many  patrollers  in 
xQuoting  from  Daily  Mail  Online,  November  11th,  2007,  “American  military  chiefs  have  been  left  dumbstruck  by 
an  undetected  Chinese  submarine  popping  up  at  the  heart  of  a  recent  Pacific  exercise  and  close  to  the  vast  U.S.S. 
Kitty  Hawk  -  a  1,000  ft  super  carrier  with  4,500  personnel  on  board.  By  the  time  it  surfaced  the  160  ft  Song  Class 
diesel-electric  attack  submarine  is  understood  to  have  sailed  within  viable  range  for  launching  torpedoes  or  missiles 
at  the  carrier,”  by  Matthew  Hickley. 
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a  narrow  channel.  In  that  case,  the  limited  turn  radius  of  a  patroller  may  force  it  to  deviate  greatly 
from  the  assigned,  say,  back-and-forth  trajectory.  We  refer  the  reader  to  [1]  for  a  broad  review  of 
other  problems  in  search  theory. 

In  this  study,  we  consider  one  or  more  patrollers,  account  for  turn-radius  limits  and  imperfect 
sensors,  and  model  the  motion  of  the  patrollers  using  ordinary  differential  equations.  This  formu¬ 
lation  leads  to  an  optimal  control  problem  with  solution  trajectories  that  are  executable  by  UUVs. 
Optimal  control  formulations  of  general  search  problems  are  found  in  [3]  with  later  generalizations 
in  [9];  see  also  references  therein.  However,  these  studies  deal  with  the  general  situation  where  the 
intruder  moves  according  to  some  diffusion  process.  We  take  advantage  of  the  special  structure  of 
the  channel  patrol  problem  and  derive  significantly  simpler  expressions,  which  allow  us  to  carry 
out  a  comprehensive  numerical  investigation  of  one,  two,  and  three  patrollers. 

In  Section  2  we  derive  a  formula  for  the  detection  probability,  in  Section  3  we  present  the  opti¬ 
mal  control  formulation  of  the  channel  patrol  problem,  and  in  Section  4  we  discuss  a  discretization 
scheme  for  this  optimal  control  problem.  Numerical  results  are  found  in  Section  5,  which  is  followed 
by  our  concluding  remarks  in  Section  6. 

2  Detection  Probability 

We  consider  a  scenario  of  patrolling  a  channel  similar  to  the  one  in  [14]:  patrollers  search  a  channel 
of  width  L  looking  for  a  single  intruder  which  is  moving  straight  down  the  channel  with  constant 
speed  vi  (see  Figure  1).  The  intruder  is  unaware  of  the  patrollers,  makes  no  attempt  to  evade  them, 
and  simply  progresses  straight  down  the  channel. 

We  assume  that  the  probability  of  detection,  of  the  intruder  by  a  patroller,  depends  on  the 
positions  of  the  patroller  and  the  intruder,  the  quality  of  the  patroller’s  sensor,  and  on  the  time 
allowed  for  observation.  We  also  could  easily  let  the  probability  of  detection  depend  on  the  speeds  of 
the  intruder  and  patrollers,  but  ignore  that  possibility  here  to  avoid  complicated  detection  models. 
(We  do  explore  one  effect  of  variable  intruder  speed  in  Section  5.) 

Suppose  that  there  are  q  patrollers  looking  independently  for  the  intruder  and  that  Xfc(t)  = 
x|(i))  £  R2  is  the  position  of  the  fc-th  patroller  at  time  f,  k  =  1,2,  see  Figure  1  for  the 
case  with  q  =  2.  We  use  superscripts  to  denote  components  of  a  vector.  Of  course,  UUVs  can  also 
vary  their  depth,  but  we  ignore  this  possibility  for  simplicity  of  exposition.  The  formulation  below 
can  trivially  be  extended  to  three  dimensions. 

We  derive  the  expression  for  the  probability  of  detection  in  two  steps.  First,  we  derive  the 
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Figure  1:  Two  patrollers  (bottom)  try  to  detect  an  intruder  (top)  in  a  channel 

detection  probability  for  a  stationary  intruder.  Second,  we  extend  that  expression  to  the  situation 
at  hand  with  a  moving  intruder  in  a  channel.  Hence,  temporarily  assume  that  the  intruder  is 
stationary  and  located  at  y  E  M2.  Again,  an  extension  of  the  following  formulation  to  three 
dimensions  is  trivial.  Let  rk{xk{t),y,t)  >  0,  k  =  1,2 denote  the  detection  rate  at  time  t 
for  the  fc-th  patroller  at  xk(t )  when  the  intruder  is  located  at  y.  The  detection  rates  reflect  the 
qualities  of  the  patrollers’  sensors  as  described  in  more  details  below  and  are  defined  so  that  the 
probability  that  the  k- th  patroller  detects  the  intruder  during  a  small  time  interval  [t,t  +  At) 
is  rk(xk(t),y,t)At.  For  theoretical  and  computational  reasons,  rk(-,-,-),  &  =  1,2,  must  be 
smooth,  but  can  otherwise  take  any  form  to  reflect  a  variety  of  sensors. 

We  focus  on  patrollers  that  are  UUVs  and  intruders  that  are  diesel-electric  submarines,  and 
assume  that  the  patrollers’  sensors  are  sonars.  Hence,  we  adopt  the  Poisson  Scan  Model  (see,  e.g., 
[13]  p.  3-1)  and,  for  the  k- th  patroller,  we  set 

rk(xk(t),y,t)  =  A<L[{Ffc  -  p(xk(t),y)}/a\,  (1) 

where  <!>(•)  is  the  standard  normal  cumulative  distribution  function,  A  is  the  scan  opportunity  rate, 
Ffc  is  the  “figure  of  merit”  (a  sonar  characteristic),  a  reflects  the  variability  in  the  “signal  excess,” 
and  p(xk(t),y)  is  the  propagation  loss,  which  depends  on  the  distance  between  the  patroller  and 
the  intruder,  see,  e.g,  Figure  4.5  on  page  93  in  [12].  All  these  quantities  may  be  time  dependent. 
The  typical  shape  of  rk(xk(t),  •,  t)  is  shown  in  Figure  2,  where  x(t)  =  (0,0)  and  p(xk(t),y )  = 
a\\xk(t)  —  y ||2  +  b,  with  A  =  1,  Fk  =  70,  a  =  5,  a  =  0.5,  and  b  =  60.  We  now  define  the  probability 
that  the  k- th  patroller  does  not  detect  the  intruder  during  some  time  interval  [0,  T]  in  terms  of  the 
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Figure  2:  Detection  rate  function  based  on  Poisson  Scan  Model  (1). 


detection  rate. 

Given  a  trajectory  {xk(t),  0  <t<T}  and  an  intruder  at  y.  we  denote  the  probability  that  the 
k-th  patroller  does  not  detect  the  intruder  during  [0,  t],  t  G  [0,  T],  by  pk(y ,  t ).  Assuming  that  events 
of  detection  in  non-overlapping  time  intervals  are  all  independent,  we  find  that  this  probability  can 
be  computed  recursively  by  solving  the  difference  equation 

Pk(y,t  +  At)  =  pk(y,t)  (1  -  rk(xk(t),y,t)At) ,  pk(y,  0)  =  1,  (2) 

or,  as  At  tends  to  zero,  by  solving  the  parameterized  differential  equation 

=  ~Pk(y,t)rk(xk(t),y,t),  pk(y,  0)  =  1,  (3) 

with  solution 

Pk(y,  t )  =  exp(— rjk(y,  t)),  (4) 

where 

Vk(y,t)=  rk(xk(s),y,s)ds.  (5) 

Jo 

The  above  derivation  follows  standard  arguments  for  Poisson  processes  and  %(?/,£)  is  the  mean 
value  of  the  random  number  of  detections  at  y,  up  to  time  t,  by  the  k- th  patroller,  when  that 
number  is  given  by  a  Poisson  law. 

Now,  let  cj)  :  M2  — >  R  be  the  probability  density  function  of  the  location  of  the  (stationary) 
intruder  at  time  0,  i.e.,  for  any  B  C  M2,  JB<j>(y)dy  is  the  probability  that  the  intruder  is  located 
in  the  area  B  at  time  0.  This  information  may  be  provided  by  exogenous  intelligence  sources  and 
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reflects  the  patrollers  knowledge  about  the  intruder  prior  to  the  start  of  the  patrols.  Then,  the 
probability  that  the  k- th  patroller  fails  to  detect  a  stationary  intruder  during  the  time  period  [0,  T] 
is  given  by 


Pk{y,T)<t>(y)dy 


(6) 


=  /  exp  -  /  rk(xk(t),y,t)dt)  <f>(y)dy.  (7) 

Jye R2  V  Jo  J 

The  functions  pk(-,t),  k  =  1,2 reflect  the  patrollers’  knowledge  about  the  intruder’s 
location  at  time  t  and  can  therefore  be  considered  to  be  “information  states”  or  “belief  states”  that 
augment  the  “physical  state”  xk(t),  k  =  1,2, 

The  extension  from  a  stationary  intruder,  as  assumed  above,  to  an  intruder  that  moves  straight 
down  a  channel  at  constant  speed,  see  Figure  1,  is  accomplished  by  a  linear  transformation  as 
described  next. 

As  in  [14],  we  fix  the  position  of  the  intruder  on  a  tape  moving  down  the  channel  at  the  speed 
of  the  intruder,  vj.  Hence,  the  intruder  is  stationary  relative  to  the  tape  and  the  formulae  derived 
above  are  applicable.  We  only  need  to  measure  the  patroller’s  location  relative  to  the  tape.  In 
this  framework,  the  probability  of  detection  relates  to  the  ratio  of  the  rate  at  which  the  patroller 
examines  new  area  on  the  tape  to  the  rate  at  which  new  tape  area  appears. 

In  order  to  utilize  this  approach,  let  zk(t )  =  {z\(t) ,  z^{t))  be  the  position  vector  of  the  k- th 
patroller  at  time  t  relative  to  the  tape.  Then  we  have  that  for  all  k  =  1,2, 


4(t)  =  x\(t) 


(8) 


44)  =x\{t)+VIt. 

We  refer  to  xk(t)  and  zk(t)  as  the  absolute  and  relative  positions  of  the  fc-tli  patroller  at  time 
t,  respectively.  We  will  use  y  for  both  the  absolute  and  relative  positions  of  the  intruder  as  the 
meaning  is  clear  from  the  context. 

Since  the  channel  has  width  L,  it  suffices  to  consider  relative  intruder  position  y  6  A(T)  = 
[0  ,L\  x  [0,  viT]  for  patrols  of  duration  T  time  units.  Hence,  it  follows  from  (7)  that  given  a  trajectory 
{zk(t),  0  <  t  <  T},  the  probability  that  the  fc-tli  patroller  does  not  detect  the  intruder  during  time 
period  [0,  T]  is 


Pk  ~  [  exp  (-  [  rk(zk(t),y,t)dt]  <j>(y)dy,  (9) 

JyeA(T)  \  Jo  J 

where  the  probability  density  function  of  the  relative  position  of  the  intruder  takes  the  specific  form 
(f>{y)  =  01(y1)/(u/T),  with  being  the  probability  density  function  of  the  intruder’s  ^/-position 
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(i.e. ,  the  intruder’s  horizontal  position  in  Figure  1).  For  example,  if  the  patrollers  have  no  prior 
knowledge  of  the  ^-position  of  the  intruder,  then  one  can  assume  a  uniform  distribution  across  the 
channel,  i.e.,  (jYiy1)  =  1/L  for  all  y 1  G  [0,  L\.  Note  that  we  abuse  the  notation  rk(-,  -,  •)  slightly,  by 
using  it  to  represent  the  detection  rate  function  both  in  the  absolute  and  in  the  relative  positions. 

We  assume  that  the  patrollers  make  independent  detection  attempts  and  hence  it  follows  from 
(4)  and  (5)  that  the  conditional  probability  that  no  patroller  detects  the  intruder  given  a  specific 
relative  intruder  position  y  is  simply  the  product 


n  exp 

k=\ 


/  rk(zk(t),y,t)dt\  =  exp  -  V]  /  rk(zk(t),y,t)dt 
Jo  J  \  k=1J o  j 

T  1  ^ 

y^rk(zk(t),y,t)dt 
k=  1 


(10) 


=  exp 

Consequently,  the  probability  that  no  patroller  detects  the  intruder  during  [0,  T]  takes  the  form 

exP  l  -  /  7  ;rk{zk{t),y,t)at  |  <p{y)dy.  (11) 


P  = 


>y&A{T) 


rT  9  \ 

5 \2rk{zk(t),y,t)dt  J  <f){y)dy. 
3  k= 1 


We  use  this  expression  in  an  optimal  control  problem  for  determining  patrol  trajectories  as 
discussed  next. 


3  Optimal  Control  Problem 

Our  objective  is  to  find  optimal  closed  trajectories  for  multiple  patrollers  that  maximize  the  prob¬ 
ability  of  detection  of  the  intruder.  In  contrast  to  [14],  we  consider  multiple  patrollers  whose  turn 
radius  is  constrained  by  their  dynamics,  in  differential  equation  form,  and  available  control  action. 
Thus  we  assume  that  the  positions  of  the  patrollers  are  states  of  a  differential  equation.  Specifically, 
we  assume  that  the  kinematic  equations  of  all  the  patrollers  are  the  same  and  are  of  the  form 

=  f(xk(t),uk(t)),  xfc(0)  =  4,  (12) 

where  the  state  xk(t)  G  Mna:,  the  control  uk(t)  G  Mn“,  /  :  Wlx  x  Mn“  — ►  l"1  is  locally  Lipschitz 
continuous,  and  £k  is  the  initial  condition  of  patroller  k.  We  assume  that  the  first  two  components 
of  the  state,  (xk(t) ,  xk(t)) ,  represent  the  absolute  location  of  the  k- th  patroller.  Hence,  xk(t)  = 
(. xk(t )',  xk(t),xk(t), ...,  xnk’:  (t))' ,  where  prime  denotes  the  transpose  of  a  vector.  The  assumption  that 
all  patrollers  are  governed  by  the  same  kinematic  equation  is  easily  relaxed,  but  requires  further 
notation  and  is  therefore  avoided  here. 
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Next,  referring  to  (8),  let  e2  —  (0, 1,  0, . . . ,  0)  6  M”*,  and  let  zk(t)  =  xk{t )  +  vite 2.  Hence, 
zk(t )  =  (zk(t)',  x%.(t),  ...,  x^x  (t))'.  We  refer  to  xk(t)  and  zk(t)  as  absolute  and  relative  states 

for  the  fc-th  patroller,  respectively.  Then  we  find  that  the  £:-th  patroller’s  dynamics  in  the  relative 
state  become 

—^jT-  =  f(zk(t),uk(t)),  zk{  0)  =  £fc,  (13a) 

where 

f(zk(t),uk(t))  =  f(zk(t)  -  vite2,uk(t))  +  u/e2.  (13b) 

We  let  the  patrol  duration  T  be  a  decision  variable.  Hence,  we  introduce  the  time  transforma¬ 
tion  t  =  T.s  to  enable  us  to  define  the  channel  patrol  problem  on  the  fixed  time  interval  [0, 1].  For 
simplicity  of  notation,  we  use  the  same  notation  for  states  and  controls  defined  on  [0,  T]  as  on  the 
normalized  time  interval  [0, 1].  The  meaning  should  be  clear  from  the  context.  We  now  obtain  the 
time-normalized  kinematic  equations 

=  Tf(zk(s),uk{s)),  zk(  0)  =  (14) 

We  denote  the  solution  of  (14)  by  zk(--,T,uk,£k),  as  it  clearly  depends  on  the  control  input 
{uk(s),s  E  [0,1]},  the  time  horizon  T,  and  the  initial  condition  £k.  Since  the  relative  location 
zk(t)  of  the  fc-th  patroller  is  given  by  the  first  two  components  of  zk(-’,T,uk,£k)  evaluated  at 
t/T ,  it  also  depends  on  {uk(s),s  E  [0,1]},  T,  and  £k.  Moreover,  the  probability  P  that  no  pa¬ 
troller  detects  the  intruder  during  the  interval  [0,  T]  (see  (11))  is  a  function  of  T  and  the  relative 
locations  {zk(t),t  E  [0, T]},  k  =  1,2 Consequently,  P  depends  on  T,  {uk(s),s  E  [0,1]} 
and  k  =  1,2,. ,.,q,  and  to  emphasize  this  dependence  we  write  P(T,u,£)  instead  of  P,  where 
u  =  (ui,u2,...,uq)  and  £  =  (6, 6,  •  •  • ,  6?)- 

The  optimal  periodic  patrol  problem  (OPPP)  consists  of  maximizing  the  probability  of  detect¬ 
ing  the  intruder  during  the  time  interval  [0, T],  i.e. ,  1  —  P(T,u,£),  by  choosing  the  best  values  of 


T,  u,  and  £.  This  leads  to  the  following  optimal  control  problem  formulation: 

OPPP  :  max{l  —  P(T,  «,£)}  (15a) 

s.t.  zk(l;T,uk,£,k)  =g(€k),  k  =  1,2,  ...,<?,  (15b) 

zk(s-,T,uk,£k)  <  4nax(s;T),  fc  =  l,2,...,g,s  G  [0,1],  (15c) 

zk(s;T,uk,£k)  >  z^m(s;T),  k  =  1,2, ...,  q,s  E  [0, 1],  (15d) 

TG  [Tmin,Tmax],  (15e) 

u  G  U,  (15f) 

£  G  X,  (15g) 

where  g  :  R71*  — >  Rna:  is  a  function  that  describes  the  end-state  constraints,  2™ax(s;  •)  :  R  — >  R  and 
4mn(s;  •)  :  R  — »  R  are  upper  and  lower  bounds  on  the  state  trajectories  at  scaled  time  s,  respectively, 
Tmm  and  Tmax  are  the  minimum  and  maximum  durations  of  a  patrol,  respectively,  U  is  the  set  of 
admissible  controls,  and  X  C  Wlx  x  •  •  •  x  R”*  is  the  set  of  admissible  initial  conditions.  We  assume 
that  U  is  a  convex  subset  of  the  g-dimensional  Cartesian  product  T)^2[0,  1]  x  •  •  •  x  LSd“,2[°’  !]>  where 
L£2[0, 1]  denotes  the  pre-Hilbert  space  whose  elements  are  functions  from  [0, 1]  to  Rn“,  which  are 
in  L^[ 0, 1],  i.e,  are  sup-norm  bounded,  with  inner  product  {111,112)2  —  fo(ui(t),  U2(t))dt  and  norm 
||  •  1 1 2  defined  by  1 1 rz. 1 1 2  =  (u,  u)^2 .2 

Specifically,  we  let 

U  =  {u  =  (■ ui,u2,...,uq )  |  uk  G  L^2[0,l],ufm  <  uk(s)  <  «“ax,Vs  G  [0,1],  k  =  1,2  (16) 

where  n™111  and  n™ax  are  the  minimum  and  maximum  control  input  at  any  point  in  time  for  the 
k-th  patroller. 

We  use  the  constraints  (15b)  to  ensure  that  the  patrollers’  trajectories  are  closed.  The  con¬ 
straints  (15c)  and  (15d)  are  set  up  to  contain  the  trajectories  of  the  patrollers  to  be  within  a  time- 
varying  box.  The  constraint  (15e)  limits  the  duration  of  a  patrol.  The  constraints  (15f)  and  (15g) 
ensure  that  the  control  input  and  initial  conditions  satisfy  specific  constraints.  We  note  that  the 
dynamics  (14)  are  implicitly  accounted  for  through  the  definition  of  P{T,u ,  £)  and  zk(-;uk,T,£k), 
k  =  1,2 

We  replace  the  “running  cost”  exp(—  J2k=i  rk(zk(t),  y,  t)dt)  in  (11)  with  an  “end  cost” 
using  an  auxiliary  information  state  p(y,  s )  to  facilitate  the  evaluation  of  this  integral  by  the  same 
2The  reason  for  using  this  “hybrid”  space  is  that  our  cost  and  constraint  functions  are  differentiable  on  L7^ 2[0, 1], 
but  they  are  not  necessarily  differentiable  on  the  well-know  space  [0, 1]  of  Lcbesgue  square-integrable  functions 
with  the  same  scalar  product  and  norm;  see  Section  5.6  in  [10]. 
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numerical  integration  technique  used  to  solve  the  dynamic  equations  (14).  For  any  y  6  R2,  let 
p(y ,  s)  be  the  solution  of  the  parameterized  differential  equation 


dp{y, f) 

ds 


-Tp(y,s)^2rk(zk(s),y,Ts),  p{y,  0)  =  1. 


(17) 


k= 1 


In  view  of  (3),  p(y,  s )  is  the  probability  that  no  patroller  has  detected  the  intruder  during  the  time 
interval  [0,  Ts]  given  the  intruder  is  located  at  y.  It  generalizes  the  information  state  pk{y,t )  to  the 
case  of  multiple  patrollers,  relative  locations,  and  scaled  time. 

In  this  notation, 


P(T,u,£)=[  p{y,  l)(f)(y)dy,  (18) 

JyeA 

where  p(y,  1)  is  given  by  (17)  and  computed  using  T ,  u,  and  £,  and  A  =  [0,  L]  x  [0, vi\.  Note 
that  similarly  to  the  change  from  the  time  interval  [0,T]  to  the  scaled  time  interval  [0, 1],  the  area 
A(T)  =  [0,  L\  x  [0,  viT ]  is  replaced  by  the  scaled  area  A. 

The  numerical  solution  of  OPPP  requires  the  discretization  of  the  time  interval  [0, 1]  and  of 
the  area  A,  as  we  describe  in  the  next  section. 


4  Discretization 


We  consider  the  time  and  space  discretizations  in  turn.  First,  we  deal  with  the  discretization  of 
the  rectangular  area  A,  using  a  N\  by  JV2  grid  defined  by 

yj  =  iAi  and  y 2  =  jA2,  (19) 


where  Ai  =  L/N\,  A2  =  vj /7V2 ,  i  =  0, 1, . . . ,  Ni,  and  j  =  0, 1, ... ,  1V2.  We  also  define  center  points 


of  the  grid  by 


,(*b) 


Vi  ~  Ai/2 

y)  -  A2/2 


(20) 


for  i  =  1, 2, . . . ,  N\  and  j  =  1,2,...,  7V2. 

Let  Pij(s)  —  p(yc’^\  s).  Then,  for  the  center  points  of  this  grid,  (17)  becomes 

dplJ['^  =  —Tpij(s)  rk{zk(s),y^’3\Ts),  pij{ 0)  =  1. 


(21) 


k= 1 


Our  approach  works  with  any  spatial  discretization  scheme.  For  example,  if  the  given  problem 
is  to  find  a  patrolling  trajectory  inside  a  closed  area  with  an  arbitrary  shape,  one  can  use  a  triangular 
mesh  grid. 
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Second,  we  consider  discretization  of  the  dynamics  in  time.  We  follow  the  procedure  described 
in  [10]  and  use  Euler’s  method  with  time  step  A  =  1/N,  N  a  positive  integer,  to  obtain  the 
discretized  dynamics  of  (14)  and  (21): 

zk((l  +  1)A)  -  zk(lA)  =  ATf(zk(lA),uk(lA)),  zk( 0)  =  £fe,  (22a) 

for  k  =  1,2, ...,  q  and 

q 

Pij((l  +  1)A)  -pij(lA)  =  -ATpij(lA)  ^ rk{zk{lA),y^’3\TlA),  py(0)  =  1,  (22b) 

k= 1 

for  i  =  1,2,... ,  iVi  and  j  =  1,2,...,  IV2,  with  l  =  0, 1, . . . ,  N  —  1. 

Third,  we  discretize  the  control  input  u(-).  For  any  /  =  0, 1,  2, ...,  N  —  1,  we  define 

ui  =  •  •  •  u'lqy,  (23a) 

where  uitk  €  M.Nu  is  the  control  input  for  the  fc-th  patroller  at  scaled  time  l A,  k  =  1,2,  Also, 
let 

u  =  (23b) 

and  for  any  k  =  1,  2, ...,  q  let 

h.,fc  =  (^0 ■  (23c) 

To  ensure  norm-preservation  between  the  infinite-dimensional  input  u{- )  and  the  discretized  input 
u ,  we  scale  h  with  the  time-discretization  level  and  let  uk{lA)  =  '/Nuijk  for  all  1  =  0,1, ...,  N  —  1 
and  k  =  1,2,  ...,g;  see  pp.  722-723  in  [10]. 

Finally,  let  z/j(?  be  the  fc-tli  patroller’s  approximate  state  at  time  step  l  when  using  both  the 
discretized  dynamics  (22a)  and  the  discretized  input  (23b).  That  is,  for  any  for  k  =  1,2,  ...,q  and 
for  l  =  0, 1, ...,  N  —  1,  let 

zi+ i,fc  -  ziyk  =  ATf(zi,k,VNui,k),  z0,k  =  £k-  (24a) 

Similarly,  let  pi^j  be  the  approximate  probability  that  no  patroller  has  detected  the  intruder  up  to 

(i  j) 

time  step  l,  given  that  the  intruder  is  located  in  the  discretized  area  represented  by  yc  .  Then  we 
see  that  puj  satisfies  the  difference  equation, 

q 

Pi+i,ij  ~  Pi,ij  =  -ATpi^j  ^2  rk{zi,k ,  V^iTlA),  Pij( 0)  =  1,  (24b) 

k= 1 

for  i  =  1,  2, . . . ,  Ni  and  j  =  1,2, ... ,  AT2,  with  l  =  0, 1, . . . ,  N  —  1.  Here  z^k  denotes  the  first  two 
components  of  zitk- 
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We  emphasize  that  zi.k  depends  on  T,  u.^,  and  by  writing  ^(T,  u.^,  £&)  instead  of  zi tk- 
In  view  of  (18),  the  approximation  of  P(T,u ,  £),  denoted  by  Pn,nl,n2(T,u,^),  using  the  above 
discretization  scheme  takes  the  form 

iVi  n2 

Pn,N!,n2(T,u,$,)  =  (25) 

i= 1  3= 1 


Hence,  for  any  positive  integers  N .  N\ ,  and  A^,  the  time-and-space  discretization  of  OPPP 
takes  the  form 


OPPP  (TV,  TVi,  iV2)  :  max  { 1  -  PN,Nl,N2(T,  u,  £)} 
s.t.  zN,k(T,u.,k,£,k)  =  g(£k),  k  =  l,2,...,q, 

zi,k(T ',  4)  <  2z“ax(T),  A:  =  1,2, ...,  q,  l  =  0, 1, ...,  IV, 

6)  >  ^“m(T),  k  =  !> 2>  9,^  =  0, 1, IV, 

rjy  ^  jjimin  2"’maxJ 

G  [u^/Vn,u^/Vn\,  k  =  1,2,  ..,g,Z  =  0, 1  -  1, 

^X, 


(26a) 

(26b) 

(26c) 

(26d) 

(26e) 

(26f) 

(26g) 


where  z'1a^x(T)  =  z™ax(ZA;T)  and  z™™{T)  =  z™in(/A;T).  The  constraints  (26b)-(26d)  and  (26f) 
are  discretized  versions  of  the  corresponding  constraints  in  OPPP. 

The  problem  OPPP(IV,  TVi,  N2)  has  a  large  number  of  decision  variables,  and  the  dimension 
of  the  underlying  augmented  discrete  dynamics  (24a)  and  (24b)  is  also  large.  Specifically,  the 
dimension  of  the  dynamics  is  nxq+N\N2 ,  and  the  number  of  decision  variables  in  OPPP(IV,  IVi,  IV2) 
is  Nnuq  +  nxq  +  1. 

To  solve  OPPP(A,  IVi,  IV2),  one  can  use  collocation  methods  [2],  which  treat  the  control 
and  the  state  as  independent  variables.  Although,  in  this  case,  the  gradient  computations  become 
relatively  simple,  the  resulting  nonlinear  programming  problem  has  a  large  number  of  variables  and 
a  large  number  of  nonlinear  (collocation)  equality  constraints,  representing  the  dynamics.  Since  the 
dimension  of  the  augmented  discrete  dynamics  is,  normally,  quite  large,  using  collocation  methods 
would  result  in  serious  numerical  difficulties  unless  a  solver  specialized  in  dealing  with  a  large 
number  of  sparse  collocation  constraints  is  used.  The  pseudospectral  method,  also  known  as  the 
orthogonal  collocation  method  [5]  may  reduce  the  size  of  N,  and  therefore  the  number  of  variables 
and  discretized  constraints.  However,  it  has  only  been  validated  for  the  solution  of  optimal  control 
problems  with  continuous  optimal  controls,  but  our  patrolling  problem  results  in  discontinuous 
optimal  controls.  Hence  we  prefer  to  use  the  method  presented  in  Chapter  4  of  [10],  which  regards 


12 


only  control  inputs,  initial  conditions,  and  end  time  as  decision  variables.  Numerical  results  based 
on  this  approach  are  presented  in  the  next  section. 

5  Numerical  Results 

In  the  following  numerical  examples,  we  assume  that  the  k-th  patroller’s  absolute  state  xk{t)  = 
0 4(*)  4(0  4)'  e  r3>  i-e->  nx  =  3,  where  x\(i))  represent  the  absolute  location  of  the  k- th 

patroller,  as  before,  and  x\  represents  its  heading.  We  assume  that  all  patrollers  move  at  constant 
speed  v.  The  control  input  for  the  k- th  patroller  uk  £  M  is  its  yaw  rate,  i.e. ,  nu  =  1.  This  leads  to 
kinematic  equations  in  (12)  defined  by 

v  cos  (t) 

f(xk(t),Uk(t))  =  usin  xl(t)  ,  (27) 

uk(t) 

k  =  1,2, This  planar  kinematic  model  describes  underwater  vehicles  that  navigate  at  a 
constant  depth  and  a  constant  forward  speed  with  variable  yaw  rate.  In  [6],  a  similar  model  was 
suggested  for  use  with  underwater  vehicles,  but  they  regarded  the  vehicle’s  yaw  rate  as  a  function 
of  vehicle’s  forward  speed  and  steering  angle. 

After  transformation  to  the  relative  state  form  in  (13a),  we  obtain  that 

v  cos  zf.(s) 

f(zk(s),uk(s))  =  usin  zl(s)  +  vj  •  (28a) 

Uk{s) 

In  OPPP,  for  every  patroller  k  =  1,2, ...,(/,  we  let  end-state  constraint 

a(tk)  =  +  vi,  +  2n7r)'  (28b) 

for  some  n  =  0, 1,  2, ....  This  ensure  that  the  absolute  location  and  heading  of  the  patroller  at  time 
T  is  the  same  as  at  time  0.  The  integer  n  is  a  variable  that  determines  the  number  of  360-degree 
rotations  that  are  required  during  a  patrol  and  hence,  as  we  will  shortly  see,  it  largely  determines 
the  shape  of  the  trajectory.  Since  we  cannot  deal  with  mixed  integer  programming,  we  will  resolve 
the  problem  for  n  =  0, 1,  2, ....  In  fact,  it  soon  becomes  apparent  that  one  only  needs  to  consider 
the  values  n  =  0, 1. 

We  set  the  state-trajectory  constraints  2™in(.s;T)  =  (0 ,vj.sT  —  7,-00/  and  z™ax(s::  T)  = 
( L,visT  +  7,00/  for  k  =  1,2 ,...,(7,  where  7  >  0  is  a  constant  that  we  vary  below.  We  note 
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Case 

n 

7 

pm&x. 

rj~ 1* 

JD* 

1 

0 

L/10 

25 

24.001 

0.43348 

2 

1 

L/ 10 

25 

23.568 

0.43300 

3 

2 

L/10 

25 

25.000 

0.43243 

4 

0 

L/5 

15 

15.000 

0.42462 

5 

1 

L/5 

15 

15.000 

0.42620 

Table  1:  Summary  of  numerical  results  for  a  single  patroller  and  varying  number  of  rotations  n  (see 
(28b)),  vertical  range  7,  and  patrol-duration  limit  Tmax.  T*  and  P*  are  optimized  patrol  duration 
and  probability  of  detection,  respectively. 

the  state-trajectory  constraints  imply  that  2^(s)  G  [0,  L\,  i.e. ,  the  patrollers  stay  within  the  chan¬ 
nel  and  z%(s)  G  [— 7  +  visT,  7  +  visT],  i.e.,  x\ (t)  G  [—7,7]-  Hence,  the  last  constraint  limits 
how  much  the  patrollers  can  travel  up  and  down  the  channel.  The  control  input  limits  it™ax  =  1 
and  «™in  =  —1  for  k  =  1,2,  We  let  the  constraint  set  on  the  initial  conditions  be  given  by 
X  =  {£  G  K3  |  0  <  C1  <  L,  £2  =  0,  £3  G  R}. 

We  set  the  channel  width  L  =  20,  where  one  unit  of  length  equals  1000  yards,  and  the  intruder 
speed  vj  =  3,  and  the  patroller  speed  v  =  1.  We  assume  that  one  unit  of  time  equals  0.1  hours. 
Hence,  the  intruder  and  patrollers  move  at  approximately  15  knots  and  5  knots,  respectively.  We 
always  use  Tmm  =  5  and  hence  we  do  not  consider  patrols  of  shorter  duration  that  0.5  hours.  We 
vary  Tmax.  We  use  the  detection  rate  function  (1)  with  parameters  as  given  below  that  equation. 
Hence,  the  detection  rate  function  is  as  in  Figure  2.  If  not  stated  otherwise,  we  assume  that  the 
distribution  of  the  intruder’s  ^-location  is  uniform,  i.e.,  ^>1(y1)  =  1/L.  We  set  the  discretization 
levels  with  N  =  128  and  N\  =  N\  =  32.  For  the  above  parameter  values,  the  augmented  discrete 
dynamics  are  of  dimension  1027,  1030,  and  1033  for  one,  two,  and  three  patrollers,  respectively.  The 
number  of  decision  variables  is  132,  263,  and  394  for  one,  two,  and  three  patrollers,  respectively. 

Finally,  we  use  SNOPT  version  6.2  [8]  in  TOMLAB  MATLAB  toolbox  [4]  as  our  nonlinear 
programming  solver,  running  on  a  desktop  computer  with  two  AMD  Opertron  2.2GHz  processors 
with  8GB  RAM,  running  Linux  2.6.28.  We  use  SNOPT  default  parameters. 

Next  we  describe  the  results  of  several  numerical  studies  involving  one,  two,  and  three  pa¬ 
trollers. 
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5.1  One  Patroller 


Table  1  provides  numerical  results  for  a  single  patroller,  i.e. ,  q  =  1,  for  several  values  of  the  number 
of  rotations  n  (see  (28b)),  vertical  trajectory  constraint  7,  and  maximum  patrol  duration  Tmax. 
In  cases  1-3,  7  =  L/10  =  2,  i.e.,  the  patroller  cannot  move  vertically  (in  Figure  1)  more  than  two 
units  above  or  below  its  starting  point.  Moreover,  in  cases  1-3,  the  patrol  duration  is  limited  to 
Tm ax  =  25.  Case  1  requires  the  patroller  to  return  to  the  same  heading  at  the  end  of  the  patrol 
(i.e.,  no  rotation  is  allowed  and  n  =  0  in  (28b))  forcing  the  optimized  trajectory  to  have  a  “bow-tie” 
shape,  as  displayed  in  Figure  3  (solid  line).  Since  OPPP(lV,  IVi,  IV2)  may  be  nonconvex,  we  cannot 
guarantee  that  the  control  input  that  generates  this  trajectory  or  those  reported  below  are  globally 
optimal.  However,  the  optimized  control  inputs  and  corresponding  trajectories  satisfy  the  default 
stopping  criterion  of  SNOPT  and  hence  are  close  to  a  stationary  solution  of  OPPP(!V,  N±,  IV2). 
Figure  3  also  displays  the  initial  trajectory  prior  to  optimization  (dotted  line).  The  arrows  in  Figure 
3  as  well  as  all  other  figures  indicate  the  direction  of  travel  for  the  patroller.  Large  white  and  black 
triangles  denote  initial  positions  and  headings  before  and  after  optimization,  respectively.  Since  the 
patroller’s  sensor  range  is  roughly  5  units  (see  Figure  2),  the  optimized  trajectory  is  stretched  out  so 
that  the  sensor  effectively  reaches  both  sides  of  the  channel.  The  initial  trajectory  has  probability  of 
detection  0.42145  and  length  of  patrol  15,  while  the  corresponding  optimized  numbers  are  0.43348 
and  24.001  as  listed  under  T*  and  P*  in  Table  1. 

Case  2  in  Table  1  is  identical  to  Case  1  but  requires  a  360-degree  heading  change  at  the 
end  of  one  patrolling  period  (i.e.,  n  =  1).  Hence,  the  patroller  must  return  to  a  heading  shifted 
360  degrees  from  the  initial  heading,  which  excludes  a  “bow-tie”  type  trajectory,  but  is  compatible 
with  a  “racetrack”  type  trajectory.  Figure  4  shows  the  corresponding  initial  trajectory  (dotted  line, 
probability  of  detection  is  0.42587)  and  optimized  trajectory  (solid  line,  probability  of  detection  is 
0.43300).  We  note  that  the  optimized  probability  of  detection  is  slightly  worse  for  n  =  1  than  for 
n  =  0,  0.43348  versus  0.43300. 

Case  3  in  Table  1  is  identical  to  Case  1  but  requires  two  rotations  (i.e.,  n  =  2),  which  rules  out 
both  “bow-tie”  and  “racetrack”  type  trajectories.  In  this  case,  the  initial  heading  must  be  shifted 
by  720  degrees  and  hence  the  patroller  makes  two  loops  as  shown  in  Figure  5.  The  probability  of 
detection  is  again  slightly  worse  than  for  n  =  0  and  n  =  1.  Since  the  probability  of  detection  seems 
to  decrease  as  the  number  of  rotations  increases,  we  will  restrict  ourselves  to  the  problems  with 
n  =  0  and  1. 

In  the  Cases  1-3,  the  patrol-duration  limit  Tmax  was  not  active.  In  Cases  4  and  5  this  limit 


15 


Figure  3:  Case  1:  Initial  trajectory  (dotted  line)  and  optimized  trajectory  (solid  line)  of  a  single 
patroller  with  no  rotation  (n  =  0  in  (28b)).  The  arrows  indicate  direction  of  travel  for  the  patroller. 
The  white  triangle  denotes  initial  position  and  heading  before  the  optimization,  and  the  black 
triangle  denotes  the  one  after  optimization. 


Figure  4:  Case  2:  Initial  trajectory  (dotted  line)  and  optimized  trajectory  (solid  line)  of  a  single 
patroller  with  360-degree  rotation  (n  =  1  in  (28b)). 
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Figure  5:  Case  3:  Initial  trajectory  (dotted  line)  and  optimized  trajectory  (solid  line)  of  a  single 
patroller  with  720-degree  rotation  (n  =  2  in  (28b)). 

is  reduced  to  15  and  also  the  vertical  movement  restriction  7  is  relaxed  to  L/5  =  4.  We  see  from 
Table  1  that  these  changes  impose  a  restriction  on  the  patroller  and  the  probability  of  detection 
worsens.  Figures  6  and  7  show  the  resulting  trajectories.  We  see  that  the  worsened  probability  of 
detection  is  caused  by  the  fact  that  the  shorter  patrol  duration  prevents  the  patroller  from  reaching 
the  sides  of  the  channel. 

We  also  consider  a  situation  (Case  6)  where  the  distribution  of  the  intruder’s  ^/-location  is  not 
uniform.  Suppose  that  4>1( y x)  =  2 y1/L.  Hence,  we  assume  that  the  intruder  is  more  likely  to  travel 
down  the  channel  near  the  right  side  than  the  left  side  in  Figure  1.  Figure  8  shows  the  optimized 
trajectory  for  this  case  with  no  rotation  required  (n  =  0),  7  =  L/10,  and  Tmax  =  25.  We  see  that 
in  this  case  the  patroller  prefers  a  “double  figure  eight”  trajectory  close  to  the  right  side  of  the 
channel.  The  optimized  trajectory  has  duration  25.000  and  significantly  improves  the  probability 
of  detection  to  0.61374  from  the  initial  probability  of  detection  of  0.42449. 

We  return  to  the  situation  with  a  uniform  intruder  distribution  and  consider  the  effect  of 
variable  intruder  speed.  Table  2  presents  Cases  7-12  involving  different  intruder  speeds  and  numbers 
of  rotation.  We  assume  that  detection  rate  is  as  above,  even  though  a  slower  intruder  may  be 
quieter  and  therefore  harder  to  detect  under  certain  circumstance.  In  all  of  these  cases  7  =  L/10 
and  Tmax  =  25.  Rows  two  and  three  of  Table  2  restate  the  results  for  Cases  1  and  2  from  Table  1,  in 
which  the  intruder  speed  vj  =  3,  for  ease  of  comparison.  Rows  four  and  five  give  results  for  vj  =  2. 
Naturally,  as  the  intruder  speed  reduces,  the  probability  of  detection  increases,  while  the  shapes  of 
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Figure  8:  Case  6:  Initial  trajectory  (dotted  line)  and  optimized  trajectory  (solid  line)  of  a  single 
patroller  with  no  rotation  (n  =  0  in  (28b))  and  right-leaning  triangular  intruder- location  distribu¬ 
tion. 

trajectories  remain  qualitatively  similar  (Figure  9).  This  effect  is  further  observed  for  the  Cases  9 
and  10  (vj  =  1)  and  for  Cases  11  and  12  ( vj  =  0.5).  We  note  that  in  all  cases  the  constraint  of 
no  rotation  (n  =  0)  results  in  better  probability  of  detection  than  the  requirement  of  a  360-degree 
rotation  (n  =  1).  These  results  are  qualitatively  different  from  the  “idealized”  results  obtained  in 
[12],  Chapter  9,  which  do  not  account  for  turn  radius  constraints  of  the  patroller.  There  we  see  that 
a  “back-and-forth”  trajectory  similar  to  the  one  in  Figure  4  ( n  =  1),  but  with  infinitely  sharper 
turns,  is  better  than  a  “bow-tie”  trajectory  similar  to  that  in  Figure  3  ( n  =  0)  whenever  v/vj  is 
less  than  1.8.  Since  Cases  1,  2,  7-10  involve  smaller  v/vj  ratios,  the  “idealized”  results  would  lead 
to  the  conclusion  that  a  “back-and-forth”  trajectory  would  be  best.  However,  our  numerical  results 
show  that  the  bow-tie  trajectory  (n  =  0)  is  better  when  the  patroller  is  constrained  by  its  turn 
radius. 

5.2  Two  Patrollers 

Next  we  consider  two  patrollers,  i.e.,  q  =  2,  and  four  additional  cases  as  summarized  in  Table 
3.  In  all  of  these  cases  the  patrol-duration  limit  Tmax  =  25.  Rows  two  and  three  of  Table  3 
give  the  optimized  patrol  duration  and  probability  of  detection  for  no  rotation  (n  =  0)  and  360- 
degree  rotation  (n  =  1),  respectively,  using  7  =  L/10.  Figures  10  and  11  give  the  corresponding 
trajectories.  We  see  again  that  no  rotation  (Case  13)  results  in  better  probability  of  detection. 
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Figure  9:  Zoomed-in  solution  trajectories  with  varying  vj  and  n  =  0  (see  (28b)).  For  ease  of 
comparison,  the  trajectories  are  slightly  translated  so  that  the  crossing  points  of  the  trajectories 
are  at  the  origin. 


Case 

VI 

n 

rp* 

JD* 

1 

3 

0 

24.001 

0.43348 

2 

1 

23.568 

0.43300 

7 

2 

0 

23.578 

0.49725 

8 

1 

23.178 

0.49514 

9 

1 

0 

24.177 

0.65767 

10 

1 

24.434 

0.64077 

11 

0.5 

0 

25.000 

0.88680 

12 

1 

25.000 

0.86413 

Table  2:  Summary  of  numerical  results  for  a  single  patroller,  varying  intruder  speed  w/,  and  number 
of  rotations  n  (see  (28b)),  with  7  =  L/10  and  Tmax  =  25.  T*  and  P*  are  optimized  values  of  patrol 
duration  and  probability  of  detection,  respectively. 
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Figure  10  shows  that  the  optimized  trajectories  are  similar  to  “figure  eights,”  even  though  the 
initial  trajectories  are  similar  to  the  infinity  symbol.  This  effect  is  caused  by  the  narrowness  of 
the  channel.  The  two  patrollers  obtain  better  probability  of  detection  and  less  overlap  in  their 
“coverage”  by  moving  along  the  channel  instead  of  across.  The  probability  of  detection  for  the 
initial  trajectory  is  0.78003  and  improves  to  0.82037  after  optimization. 

We  observe  that  the  trajectories  in  Figure  10  are  different  for  the  two  patrollers,  which  may 
be  counterintuitive  as  the  distribution  of  the  intruder’s  ^-location  is  uniform.  Additional  calcula¬ 
tions  show  that  the  trajectories  in  Figure  10  yield  a  larger  probability  of  detection  (0.82037)  than 
patrol  plans  consisting  of  identical  but  translated  trajectories  for  both  patrollers.  If  the  right-most 
patroller  mimics  the  left-most  patroller  in  Figure  10,  but  on  the  right  side  of  the  channel,  then  the 
probability  of  detection  deteriorates  to  0.81630.  If  the  left-most  patroller  mimics  the  right-most 
patroller,  then  the  probability  of  detection  deteriorates  to  0.81472.  The  probabilities  deteriorate 
further  when  the  patrollers  carry  out  identical  but  mirror-imaged  trajectories.  These  results  provide 
new  insight  that  is  not  easily  obtained  using  the  idealized  calculations  of  [12],  Chapter  9. 

The  optimized  trajectories  of  Case  14  with  the  constraint  of  one  rotation  (i.e.,  n  =  1)  (see 
Figure  11)  yield  a  probability  of  detection  of  0.79340,  which  is  worse  than  in  Case  13  (i.e.,  n  =  0). 
We  also  examined  the  configuration  with  one  patroller  constrained  to  no  rotation  (n  =  0)  and 
the  other  one  to  a  360-degree  rotation  (n  =  1).  However,  the  resulting  probability  of  detection 
(0.81234)  is  worse  than  in  Case  13. 

Cases  15  and  16  in  Table  3  show  results  similar  to  those  for  Cases  13  and  14,  but  for  7  =  L/5. 
With  this  relaxation  of  the  vertical  movement  constraint  for  the  patrollers,  we  obtain  slightly  better 
probability  of  detection.  The  relaxation  allows  for  more  complicated  patrol  trajectories  as  shown  in 
Figures  12  and  13.  We  see  that  the  patrollers  stagger  vertically  their  trajectories  to  avoid  overlap 
and  therefore  increase  the  probability  of  detection.  While  not  easily  seen  from  Figures  12  and  13, 
the  patrollers  also  synchronize  their  progress  along  their  trajectories  so  that  when  one  patroller 
moves  to  the  left,  say,  then  the  other  tends  to  move  to  the  left  also  to  fill  the  gap  between  the 
patrollers.  Figure  14  illustrates  this  effect  by  showing  the  relative  locations  of  the  patrollers  during 
t  e  [0,  T]  for  Case  16.  Such  insight  about  the  coordination  between  multiple  patrollers  cannot  be 
reached  through  single-patroller  analysis.  The  initial  trajectories  in  Case  16  result  in  a  probability 
of  detection  of  0.77806,  which  is  improved  to  0.81594  after  optimization. 
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Case 

n 

7 

JD* 

13 

0 

L/10 

25.000 

0.82037 

14 

1 

L/10 

11.633 

0.79340 

15 

0 

L/5 

25.000 

0.82354 

16 

1 

L/5 

25.000 

0.81594 

Table  3:  Summary  of  numerical  results  for  two  patrollers,  varying  number  of  rotations  n  (see  (28b)), 
and  vertical  range  7.  T*  and  P*  are  the  optimized  patrol  duration  and  probability  of  detection, 
respectively.  For  all  cases  in  the  table  the  patrol-duration  limit  Tmax  =  25. 


Figure  10:  Case  13:  Initial  trajectories  (dotted  line)  and  optimized  trajectories  (solid  line)  of  two 
patrollers  with  no  rotation  (n  =  0  in  (28b)). 
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Figure  13:  Case  16:  Initial  trajectories  (dotted  line)  and  optimized  trajectories  (solid  line)  of  two 
patrollers  with  360-degree  rotation  (n  =  1  in  (28b))  and  relaxed  vertical  trajectory  constraint. 


Figure  14:  Case  16:  Relative  locations  z\  (t)  =  zl(t))  and  Z2(t)  =  z%(t))  for  two 

patrollers  with  absolute  location  given  in  Figure  13. 
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Figure  15:  Case  17:  Initial  trajectories  (dotted  line)  and  optimized  trajectories  (solid  line)  of  three 
patrollers  with  no  rotation  (n  =  0  in  (28b))  constraint. 

5.3  Three  Patrollers 

Finally,  we  consider  three  patrollers  briefly,  for  the  single  case  of  Tmax  =  25,  7  =  L/10,  and  no 
rotation  constraint  (n  =  0).  The  optimized  probability  of  detection  is  0.94086,  improved  from 
0.90335  for  the  initial  trajectories,  and  the  optimized  patrol  duration  is  T*  =  25.000.  Figure  15 
displays  the  initial  and  resulting  trajectories.  We  see  that  the  shape  of  each  trajectory  is  quite 
similar  to  the  ones  in  Case  13  for  two  patrollers;  see  Figure  10.  We  note  that  for  two  and  three 
patrollers  the  optimized  trajectories  tend  to  become  quite  intricate,  especially  when  the  patrollers 
are  tightly  constrained  vertically  with  7  =  L/10  and  no  rotation  is  required  (n  =  0).  This  effect  is 
caused  by  the  fact  that  multiple  patrollers  make  it  suboptimal  for  each  patroller  to  search  across 
the  whole  channel.  This  would  have  caused  substantial  overlap  between  the  patrollers  and  a  lower 
probability  of  detection.  Hence,  each  patroller  is  effectively  confined  to  a  smaller  area  of  operations. 
Even  in  the  smaller  area,  the  patrollers  tend  to  prefer  longer  patrol  durations  and  the  constraint 
T  <  Tmax  is  often  active.  Longer  patrol  durations  are  usually  preferable  as  the  constraint  that 
the  patroller’s  relative  final  state  must  match  its  relative  initial  state  (possibly  with  a  rotational 
shift)  imposes  a  restriction  on  the  patroller  and  the  longer  duration  allows  more  “free”  movement 
between  those  “boundary  conditions.” 
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6  Conclusions 


We  formulated  the  channel  patrol  problem  for  multiple  patrollers  subject  to  turn-rate  constraints 
as  an  optimal  control  problem.  In  this  problem,  the  patrollers  aim  to  maximize  the  probability 
of  detecting  an  intruder  that  travels  straight  down  a  channel  with  constant  speed.  Using  dis¬ 
cretization  of  time  and  space,  we  obtained  a  large-scale  nonlinear  programming  approximation  of 
that  problem  which  we  solved  to  obtain  an  approximately  stationary  solution  and  a  corresponding 
optimized  trajectory  for  each  patroller.  In  numerical  tests  specifically  tailored  to  one,  two,  and 
three  underwater  patrollers,  an  underwater  intruder,  different  trajectory  constraints,  and  several 
intruder  speeds,  we  found  that  simple  “back-and-forth”  trajectories  across  the  channel  are  inferior 
to  more  complicated,  optimized  trajectories.  For  a  single  patroller,  the  optimized  trajectories  tend 
to  have  the  shape  of  a  bow  tie  for  a  variable  range  of  intruder  speeds.  The  optimized  trajectory 
changes  shape  to  a  “double  figure  eight”  when  the  intruder  is  known  to  bias  its  route  to  one  side 
of  the  channel.  For  two  patrollers,  the  optimized  trajectories  also  take  the  shape  of  “double  figure 
eights,”  which  may  be  staggered  when  the  trajectory  constraints  allow  sufficient  movement  along 
the  channel.  For  three  patrollers,  the  optimized  trajectories  again  resemble  “double  figure  eights.” 
The  optimized  probability  of  detecting  an  intruder  at  15  knots  in  a  channel  of  width  20,000  yards 
using  three  patrollers  at  5  knots  with  an  imperfect  sensor  of  range  approximately  5,000  yards  is 
0.94.  That  probability  is  reduced  to  0.82  and  0.43  for  two  and  one  patrollers,  respectively. 

The  results  of  this  study  provide  new  insight,  not  easily  obtained  using  geometric  calculations, 
into  efficient  patrol  trajectory  design  for  multiple  patrollers  in  a  narrow  channel  where  interaction 
between  the  patrollers  is  unavoidable  due  to  their  limited  turn  rate.  The  insight  comes  at  a  sub¬ 
stantial  computational  cost  as  the  large-scale  nonlinear  programming  approximations  may  require 
many  days  to  solve  using  standard  hardware  and  software  due  to  expensive  function  and  gradient 
evaluations  as  well  as  poor  conditioning.  We  believe  it  is  possible  to  obtain  significant  reductions 
in  computing  times,  but  defer  such  efforts  to  future  studies. 
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